1. Field of Invention
This invention relates to processing of electromagnetic radiation measurements into images and information arrays which correspond to normal human color vision.
2. Description of Prior Art
Raw measurements of electromagnetic radiation (EMR) in three wavebands are commonly converted to color images for subsequent visual appreciation or analysis by human observers, as in photography, television, printing, etc. EMR measurements are also processed into color-coded information arrays that serve as inputs to artificial intelligence applications, such as expert systems, quality control and robotics. Separate EMR measurements of a single set of objects, i.e, a single scene, will vary with changes in the illumination intensity levels and the geometry of the measurements, i.e., the illumination and viewing angles relative to the objects. It is the prevailing consensus of physicists that information on the inherent spectral reflectance properties of objects, popularly known as their color, cannot be extracted from EMR measurements unless the illumination levels and the measurement geometry are known, or a reference standard is available. On the other hand, human vision, which usually lacks this knowledge, is characterized by a degree of color constancy which has never been satisfactorily explained. This approximate constancy of the colors humanly perceived in a single scene, despite differences in illumination and in measurement geometry, is of great value to humans in their environmental interactions. It would be highly desirable to achieve such constancy by artificial means for use in the types of applications mentioned above, i.e., for both human and artificial intelligence consumers.
The prior art related to color constancy can be conveniently divided into two categories--reflectance-oriented and empirical. The first category consists of processes designed to recover the surface spectral reflectances of objects, i.e., the percentages of incident light reflected in each wavelength or waveband. All processes in this category are mathematically based and rely on assumptions about the scene content, the illumination and the measurement geometry, which either do not always hold or are so restrictive as to be impractical for most applications. These computational approaches have been reviewed recently by Dannemiller. The most advanced is probably that of Brian A. Wandell and Laurence T. Maloney (Stanford University), U.S. Pat. No. 4,648,051, entitled COLOR IMAGING PROCESS, Mar. 3, 1987. This process claims to recover both illumination and reflectance, where both are completely unknown. However, a scientific journal article published by the inventors after the patent application specifically states that their process is restricted to "the analysis of a single image drawn from a scene with fixed geometric relations among objects, light sources, and the visual sensor array". This geometrical restriction severely limits the utility of their process, since in many potential applications the geometry frequently changes. The inventors also state in the same article, "Indeed, without restrictions on the range of lights and surfaces that the visual system will encounter, color constancy is not, in general, possible."
The second category of prior art comprises those empirical approaches which attempt to match human color perception without scientific explanation. One such approach is that of Ralph M. Evans (Kodak) (U.S. Pat. No. 2,571,697, Oct. 16, 1951), entitled METHOD FOR CORRECTING PHOTOGRAPHIC PRINTS. This process has the objective of making color prints that will be judged satisfactory by amateur photographers. The principle is to equate the overall average densities of cyan, yellow and magenta in the image so that the whole picture "integrates" or averages to gray. A second empirical approach is the well-known photographic technique of dodging or unsharp masking. The basic principle is to equalize exposure levels at high spatial frequencies by eliminating low spatial frequency contrast. The idea is based on the discovery by Ernst Mach that human vision employs a similar process. Digital variations of the process are numerous and are often called local adaptive enhancement. Land's many algorithms are versions of this approach. A third empirical approach related to the color reproduction problem is the cubic model of color space devised by Benson in 1868. This is a model of color space, based on prismatic spectra, which illustrates the humanly perceived rules of color mixing, i.e., the empirical relationships seen to result from combining either the additive primary wavebands (blue, green and red) or the subtractive primary wavebands (cyan, yellow and magenta) in different proportions.
The key elements of these three empirical approaches were combined by Clark et al in an experimental process for standardizing digital images of the earth obtained by satellite-borne sensors. The purpose was to automate image interpretation by adapting to computers the essential features of human color vision, as understood empirically.
Although the empirical approaches cited above have often received pragmatic and commercial approval, they have failed to achieve scientific credibility because of lack of physical explanation for their apparent success. In patent law terms, they have been considered by experts in the art to be inoperable, or at least unreliable. For example, an image processed as described by Clark et al. appeared on the cover of the September 1984 Global Ecology issue of Bioscience. However, the process was dismissed by Botkin et al. as follows, "Although (the) images resemble what one sees on a map, the exact relationships between the satellite instruments' response and the actual characteristics of the land surface remain a research question." According to Hunt, similar skepticism appears in the British version of the Evans patent cited above, which apparently includes the sentence, "A more pleasing effect is often produced in color prints if they are so made that instead of the color balance being correct, in which gray is printed as gray, it is so adjusted that the whole picture integrates to gray." (underlining added) Local enhancement techniques are generally considered to be ad hoc and unreliable because they are usually tailored to individual images on a subjective basis. Finally, the Benson color cube has been virtually ignored by physicists, presumably because it has lacked a theoretical explanation.
In summary, all of the color normalization processes and models heretofore suggested suffer from one or the other of two disadvantages:
(a) they are unable to recover constant images of the same scene under varying conditions of illumination intensity and measurement geometry, or PA1 (b) their success in matching human color perception cannot be explained scientifically. PA1 (a) to provide a color normalization process which has a sound scientific explanation for its operability; PA1 (b) to provide a color normalization process which produces images and information arrays of a scene which are substantially invariant with respect to both illumination intensity levels and measurement geometry; and PA1 (c) to provide a color normalization process which produces images and information arrays which agree with human color perception and description .